Comparative Study of Disabled vs. Non-disabled Evaluators in User-Testing: Dyslexia and First Year Students Learning Computer Programming

User-testing is a critical activity in software development. However, eliciting appropriate test-users can be difficult. Recent work showed that (during user-testing of educational software) dyslexic final-year students identified a larger number of subtle (yet significant) issues in more detail than other students. However, final year students were not the target users of the software (designed to teach fundamental programming concepts). This paper presents preliminary results of work replicating the previous study, but with participants from the target user group (first year students). The first year students identified fewer issues and gave less detail than the final year students. The dyslexic students identified more issues in greater detail than the other students. This highlights a distinction between the perceived target user group (first year students) and the actual target user group (students who don’t understand programming concepts). Dyslexia may push people deeper into the actual target user group for educational software.